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Building District Capacity for System-Wide Instructional Improvement in Erie Public Schools 

Executive Summary 

This report summarizes findings from one component of the Consortium for Policy Research in 
Education's (CPRE) evaluation of the General Electric Foundation's (GEF) Developing Futures ™ in 
Education program in Erie Public Schools (EPS). The purpose was to closely analyze the district's capacity 
to support system-wide instructional improvement. To understand how EPS, one of the four Developing 
Futures™ districts that were examined, built capacity for system-wide instructional improvement, our 
study during Phase Two focused on a single, overarching question: to what extent has EPS central office 
adopted and institutionalized the seven core principles of Developing Futures ™? 

This executive summary provides a brief explanation of the findings from the EPS analysis that emerged 
from the study. The analyses presented in this summary are based on interview and survey data 
gathered between January and April of 2012, The CPRE research team conducted in-person interviews 
with 19 stakeholders in EPS, including 10 central office staff members in leadership roles (including the 
superintendent), 4 principals, 3 board of education members, and 2 external partners. 

To complement and support these qualitative data, a detailed survey was administered to 22 principals 
in the spring of 2012. The survey focused largely on principals' perceptions of central office capacity, 
including clarity of vision, openness to collaboration, coherence and alignment of instructional supports, 
responsiveness to principal needs or concerns, and overall accountability. All 22 EPS principals 
completed the survey for a response rate of 100 percent. 

We studied the districts' progress in scaling up and institutionalizing the seven core elements 1 of 
Developing Futures ™: 

1. Internal constituency engagement. The district engages stakeholders at all levels of the system, 
and establishes common vision and buy-in for improvement efforts. 

2. External constituency engagement. The district engages partner organizations and institutions, 
parents and the community; and effectively communicates about reform efforts. 

3. Curriculum and instruction. The district communicates and supports a system-wide vision for 
instructional improvement. 

4. Professional development for instruction. The district delivers high-quality professional 
development on curriculum, instruction, standards, and assessment. 

5. Professional development for leadership. The district delivers high-quality professional 
development on leadership or management. 

6. Management capacity. The district collects and uses data, attracts and develops talent, and 
evaluates staff performance. 


1 These seven reform elements were identified through a review of GEF program materials and 
documentation, and through a close analyses of each district's reform trajectory over the life of the 
grant). 
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7. Evaluation. The district monitors and evaluates reform efforts. 

When we consider how the school system operated prior to the Developing Futures™ in Education 
program— that is, when we focus on its growth and development rather than its performance relative to 
an absolute standard— the progress is evident. Over the past few years, EPS has made real and 
significant progress in building capacity for system-wide instructional improvement. The instructional 
system, once fragmented, non-transparent and unaccountable, has become much more clearly 
articulated. 

Curriculum implementation in mathematics and science is more consistent. A central office 
characterized by fragmentation and silos has become increasingly collaborative, cohesive, and 
instructional^ focused. Once scattershot, professional development efforts for teachers are now closely 
aligned with instructional initiatives, require active participation, and are supported for the most part 
(albeit somewhat inconsistently) by school-level resources such as coaches and PLCs. The transition to 
common curricula, instructional frameworks, and professional development supports represents change 
for EPS. This added capacity will be especially important going forward, as EPS moves toward full? 
implementation of the CCSS. However, in order for EPS to be a high capacity district, the district needs 
furtherer development in management capacity and evaluation. 

At the time of data collection EPS was transitioning to the Common Core State Standards. EPS 
stakeholders acknowledged that the common core would be their main focus in the years to come. As 
the district moves deeper into this process, its challenge will be to continue to develop their capacity 
building for instructional improvement while adjusting to a new wave of standards and assessments, the 
flood of new curricula and materials that will accompany them, and continued shifts in how states and 
districts evaluate teachers and hold staff accountable for student learning. 

There is reason to be optimistic about the districts' progress as a result of Developing Futures™. EPS 
made real and significant gains in system-wide instructional capacity. This was most readily apparent in 
common curricula and frameworks, clearly articulated and communicated expectations about high 
quality instruction, strong, multi-level professional development systems, and use of student 
performance data to prioritize and plan instruction. This enhanced capacity directly supports the 
districts' work in preparing for the common core implementation. 

There is also more work to be done to continue that progress. Preparing staff for the Common Core is 
different from effecting the widespread changes in instruction. Data from EPS reveal lingering concerns 
about both the consistency and quality of teaching practice within and across schools. Though there was 
widespread concern about ongoing funding for professional development, which is not primarily a 
problem of support, but rather stems from an absence of fully developed feedback loops. Through 
heavy investment in student data systems and training, EPS has made progress in building capacity to 
differentiate between what is working and what is not (both at the classroom and program level). Too 
often, however, these analyses do not appear to produce decisions and actions to address 
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underperformance or to do more of what works and less of what does not. Using data to refine 
implementation of program and inform data are the next, critical step for EPS as it seeks to prepare 
teachers and students for the Common Core State Standards. 

Introduction 

This report summarizes findings from one component of the Consortium for Policy Research in 
Education's (CPRE) evaluation of the General Electric Foundation's (GEF) Developing Futures ™ in 
Education program in Erie Public Schools (EPS). As described in the CPRE proposal and research design, 
the purpose was to closely analyze district capacity to support system-wide instructional improvement. 
Specifically, this phase focused on a single, overarching question: to what extent has the district central 
office adopted and institutionalized the core principles of Developing Futures ™? To answer this 
question, this evaluation assesses the Erie Public School District's progress in scaling up and 
institutionalizing seven core elements of Developing Futures™. 

1. Internal constituency engagement. The district engages stakeholders at all levels of the system, 
and establishes common vision and buy-in for improvement efforts. 

2. External constituency engagement. The district engages partner organizations and institutions, 
parents and the community; and effectively communicates about reform efforts. 

3. Curriculum and instruction. The district communicates and supports a system-wide vision for 
instructional improvement. 

4. Professional development for instruction. The district delivers high-quality professional 
development on curriculum, instruction, standards or assessment. 

5. Professional development for leadership. The district delivers high-quality professional 
development on leadership or management. 

6. Management capacity. The district collects and uses data, attracts and develops talent, and 
evaluates staff performance. 

7. Evaluation. The district monitors and evaluates reform efforts. 

These seven reform elements were identified through a review of GEF program materials and 
documentation, and through a close analyses of each districts' reform trajectory over the life of the 
grant. Based on a thorough review of the research and evaluation literature, a set of indicators was 
constructed to allow the research team to determine the extent to which there was evidence of 
effective practice in each of these seven areas. Each area was decomposed into a set of more specific, 
observable characteristics. Research instruments were designed to elicit evidence of these 
characteristics in descriptions of central office processes, functions, or overall capacity. Ratings were 
then assigned to each characteristic based on the prevalence of available evidence using a three-point 
scale: 

1. Strong implementation. The district has reached a majority of key actors within the system. 

2. Moderate implementation. The district has reached a considerable proportion of key actors 
within the system. 
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3. Weak implementation. There is little evidence of institutionalization across the sample. 

This report provides ratings for EPS for each indicator and its component characteristics, along with 
qualitative and survey evidence illustrating and supporting the ratings. Overall, we find that EPS has 
made significant and notable progress in developing a cohesive instructional system in mathematics and 
science, including common curricula, materials and instructional frameworks. The district has also 
significantly improved its professional development , focusing more specifically on curriculum and 
instruction and building in school-level supports to complement district-level training. Major progress 
was also made on building an information technology (IT) infrastructure. 

On the other hand, it did not appear that the enhanced IT infrastructure had yet translated into 
sophisticated or extensive use of data for decision-making or evaluation. The district's talent 
management systems and practices were an area of concern; politics, relationships, and seniority rules 
appeared to dictate many hiring and placement decisions, though the administration was actively 
working to address this. Lastly, while the central office appeared adept at collecting data for the 
purposes of program design or needs assessment, there was little if any evidence of evaluation efforts 
targeting specific reform initiatives, or of those data being used to make decisions about program 
expansion or discontinuation. 

Methodology 

The analyses presented in this report are based on interview and survey data. In January 2012, the 
research team conducted in-person interviews with a diverse set of stakeholders in Erie, including 10 
central office staff members in leadership roles (including the superintendent), four principals, three 
board of education members, and two external partners. The interviews were divided into two parts. In 
the first part, respondents were asked to describe a high-priority project or initiative on which they were 
currently working. Follow-up questions focused on how the initiative became a priority, who was 
involved in its planning or implementation, how it was being implemented, and how progress was 
monitored and evaluated. The goal was to elicit evidence of the seven indicators in the context of 
current district priorities, practices, and routines. For example, if district leaders described the 
introduction of a new elementary mathematics program as a high priority, the interviewer focused on 
the extent to which those efforts were collaborative, how they were communicated and supported, 
what the intended goal was, and how progress was measured. 

All interviews were professionally transcribed. Transcripts were then coded using a deductive 
framework (that is, one that is derived from the research literature rather than being emergent from 
within the data themselves) based on the characteristics aligned with each characteristic. This allowed 
for transcript data to be sorted by indicator and specific characteristic. Finally, a participant matrix was 
constructed to generate ratings for each characteristic. For each participant and characteristic, the 
analyst indicated whether the characteristic was evident in the data, whether it was not evident in the 
data, or if no determination could be made based on the data. Characteristics that were evident in 80 
percent or more of interviews for which sufficient data were available were scored a 3, and classified as 
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strong implementation. Those that were evident in 50-79 percent of the interviews were scored a 2, and 
classified as moderate implementation, while those that were evident in less than half of the interviews 
were scored a 1, and classified as weak implementation. Due to the small size of EPS central office and 
principal sample, there were several instances in which there was insufficient data across the interviews 
to make a determination about the prevalence of a given characteristic. In these instances, applicable 
qualitative data are described but no rating is assigned. 

To complement and support these qualitative data, a detailed survey was administered to all EPS 
principals in the spring of 2012. All 22 principals completed the survey— a 100 percent response rate. 
The survey focused largely on principals' perceptions of central office capacity, including clarity of vision, 
openness to collaboration, coherence and alignment of instructional supports, responsiveness to 
principal needs or concerns, and overall accountability. The survey offered a less detailed but broader 
view of principal perceptions of the district. In the sections that follow, survey findings are reported 
alongside qualitative data for each indicator. 

Indicator 1: Internal Constituency Engagement 

As shown in Table 1, there was considerable evidence that the EPS central office was effective in 
building a broad base of support for its overall vision, engaging stakeholders throughout the system in 
planning and decision-making, and collaborating both vertically and horizontally. Principals and central 
office staff described EPS as undergoing a transformation from an organization that was hierarchical and 
siloed to one that emphasized shared decision-making and buy-in at all levels. This was a slow and 
uneven process, with progress readily apparent in some areas and slower to materialize in others. Part 
of the challenge was organizational and technical: creating opportunities for staff to collaborate, 
merging IT platforms and data systems, or defining departmental functions or individual responsibilities. 
Underlying all of these technical challenges, however, were issues of trust. Interview and survey data 
suggest that EPS has made meaningful progress in building trust and buy-in, but also that work 
remained to be done. 


Table 1. Internal Constituency Engagement 

Input is sought from internal stakeholders in planning and decision-making. 

3 

Internal stakeholders express ownership of or are "bought into" improvement 
projects or initiatives. 

3 

Horizontal collaboration (across departments) is evident. 

3 

Vertical collaboration (between levels) is evident. 

3 


Stimulated and facilitated through GEF support, the primary strategy for gathering input and 
establishing buy-in for system-wide decisions was a series of topical committees. Committees focused 
on both functional and curricular areas, with literacy and mathematics the primary curricular focus. 
Teachers, principals, and central office staff participated on committees, which had decision-making 
authority over curriculum selection/development, implementation and support. The fact that these 
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committees had real responsibility and were imbued with real authority significantly enhanced 
participants' commitment to their work. One of the goals of the early committee work was to model a 
process for engaging stakeholders and making decisions— something that appears to have taken root in 
recent years. One principal described the recent work of the science committee: 

Whenever there's a district initiative now, there's always the opportunity for teachers to 
get on the committee. There is always the opportunity. For example, science, when this 
was being created, the science teachers were able to be there, math teachers were able 
to be there. We did selection of textbooks when this was coming out. So, this wasn't a 
top down, "this is the book we like and you're going to teach it." No, they were at the 
table, they got to look at it. And I know when we did social studies, I got to sit down and 
look at the books and say, "This fits into social studies, this doesn't fit. This isn't going to 
be the best for our students and teachers." (P03) 

A central office staff member used similar language to describe the district's current work on preparing 
for the implementation of Common Core State Standards (CCSS). "As we've changed the math and the 
science and now we're moving to literacy, we realized— and this definitely came from the 
coordinators— that the building principals have to be front and center with this piece, and they have to 
know it themselves." (CO09) 

While interview data suggest that these committees have been highly effective in establishing buy-in for 
major district initiatives, survey data present a more mixed picture of the extent to which principals 
actually feel empowered to make decisions in their own buildings. Asked how much influence they have 
over hiring teachers, for example, 18 percent replied that they had "none," and 36 percent said they had 
only "a little." On spending of discretionary funds, 32 percent of principals reported having no influence 
and 46 percent having little. For determining the content of professional development for teachers in 
their buildings, 27 percent said that had no influence; an additional 32 percent said they had a little. 

Building buy-in and collaborating effectively requires trust. While involving school staff in major 
decisions had clearly helped in establishing trust, there was also some evidence that further 
improvements were needed. This was most readily apparent around the district's coaching initiative. 
When coach positions were initially created, the teachers' union stipulated that principals were not 
allowed to know which teachers were being supported by coaches— a move prompted by concern that 
working with a coach amounted to an admission of weakness and might be punished. Interestingly, a 
similar problem played out in school-central office relations, with some principals not utilizing coaches 
for fear that it would be read as a sign that their school was struggling. In both cases, challenges 
stemmed from concern that coaches were playing an unofficial supervisory role. "It's supposed to be 
coordinators, instructional coaches who are teachers, and then teachers," remarked one central office 
staff member. "They're supposed to be on the same level. And they're not. They feel like they're more in 
an administrative capacity, and that's kind of a turn-off for a teacher." (CO06) 
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Survey data also suggested persistent trust issues among principals. Presented with the statement, 
"school leaders trust central office staff," 55 percent of principals disagreed or strongly disagreed. On 
the other hand, principal interviews showed that the district was making some headway in building 
principal trust. Describing a decision to allow principals to organize and run their own meetings (rather 
than have them planned and run by the central office), one principal remarked: 

I think it gave everyone the feeling of our professionalism is respected because now 

we're not being babysat, for lack of a better term, by someone from central office. 

We're being trusted as a group of professionals to run a meeting and work together. 

And that, I think, has given everyone a feeling of, you know, trust. You know, now our 

superintendent trusts that we're professionals. It's a good feeling. (P01) 

Understanding EPS efforts to improve horizontal collaboration requires an understanding of the 
system's history. "Our organization has a long history of, I'd say, departments functioning in isolation," 
one central office staff member reflected. "And it was weird, it was very much top-down, but as far as 
central office, the various departments had a great deal of autonomy, because the senior staff wasn't 
too involved in day to day operations." (CO04) One of the first initiatives undertaken by the new 
superintendent in 2010 was a comprehensive review of departmental functions and responsibilities. This 
ultimately led to closer coordination between some functions, such as assessment and IT. Other 
departments, such as child study and special education, were merged under a single director. Along with 
the reorganization, regular cabinet-level meetings were established to ensure that departments 
communicated effectively. Together, these changes have resulted in improved coordination and 
communication across departments. 

Indicator 2: External Constituency Engagement 

As shown in Table 2, engaging local business, universities, and community organizations appeared to be 
a strong suit in Erie public schools. Three thematically oriented committees provided a system for 
generating input and feedback from external stakeholders, and school level collaborations with local 
businesses generated both volunteer and financial support. Volunteers from GE played an important 
role in building district capacity in human resources and IT, as well as working directly with students in 
some schools. Through town hall meetings, the district sought to both increase its own transparency and 
accessibility to the public while also demonstrating a desire to solicit and act on community feedback. 

A few areas of concern were also evident. While partnerships with business were strong, some 
respondents indicated that they remained insufficient to address the district's growing needs, both in 
terms of financial support and in helping high school graduates transition to careers. There were also 
indications that the district and teachers' union were not in accord on issues related to support and 
accountability at the individual teacher level, which could pose a growing concern as reforms targeting 
teacher quality become more prevalent across the state and the nation. 
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Table 2. External Constituency Engagement 

Input is sought from external stakeholders in planning and decision-making. 

3 

A communication strategy to communicate to the public about reform 
activities is in place. 

3 

The district has leveraged resources from external stakeholders (not including 
GEF) to support reform efforts. 

3 

School leaders have leveraged resources from external stakeholders to 
support reform efforts. 

3 

The district has leveraged resources from GE to support reform efforts. 

3 


There was widespread agreement that the EPS central office had become more transparent and 
accessible to the public and to external stakeholders since the new superintendent came on board in 
2010. "It's come along with Jay," commented one respondent who worked closely with the central 
office. "His open door policy is something new that no one is used to. ..There's no hidden agenda, and 
everybody's going to know everything. There's no reason not to." 

To engage external stakeholders more broadly and systematically, the superintendent created advisory 
committees focused on finance, academics, and community. The finance advisory committee has played 
a central role in helping the district negotiate recent and significant budget challenges, including a $26 
million dollar deficit in 2011. Comprised of leaders and chief financial officers from local business and 
corporate partners (including GE), the committee has provided advice and technical assistance to both 
address short-term shortfalls and the long-term structural factors which contributed to them. 

Representatives from four local institutions of higher education serve on the academic advisory 
committee. In addition to facilitating communication among all parties, the committee assisted the 
district on planning and technical issues such as assessment, as well as advising on school closings and 
grade reconfigurations dictated by the budget crisis. The committee also served as a coordinating body 
for university student placements in district schools, and ensured that those students received sufficient 
support to be effective in their roles. While some central office staff suggested that placing and retaining 
student teachers remained a challenge, it appeared that communication among the universities, the 
central office, and the schools was improving. 

The primary role of the community advisory committee is to gather input from citizens and community 
groups, relay it back to the district, and work with district staff to respond effectively. Comprised 
primarily of parents, community activists, and faith leaders, one key role of the committee is to ensure 
that groups that have traditionally had less of a voice in school or district affairs (such as low income or 
minority communities) are fairly represented. 

In 2011-12, the community advisory committee played a central role in organizing and responding to 
feedback gathered at a series of public, town hall meetings. The meetings were highly participatory: 
after brief introductory remarks from the superintendent, the participants broke into groups to identify 
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what they viewed as the highest priorities for the districts. A cross-group discussion sought to build 
consensus around these priorities, which were in turn reported back to the advisory committee. 
Specifically, the town hall meetings produced a high level of consensus about the importance of keeping 
the arts in Erie schools, and concern that testing pressure and budget cuts would further erode them. 
The district in turn committed to finding new channels for supporting arts education next year. 

The town hall meetings reflected a broader district effort to better communicate with the public, and to 
ensure that the communication was two-way. A full-time communications director was hired in January 
of 2012, and central office leaders had begun to examine more closely public perceptions of the district. 
Yet there were also indications that more work was needed in this area. Of particular concern was the 
lack of a clear strategy for communicating to the public about the transition to the CCSS. One central 
office staff member noted that familiarity with Common Core was uneven from school to school, and 
even within the central office itself, which in turn resulted in uneven communication to external 
stakeholders. 

I think we're in different places with the Common Core, and it depends on if you're 
talking about math, science, general, in terms of how we're going to work with our 
leadership and get that message out, and communicated to the community, to the 
parents, to the administrators, to the teachers. So there's pockets, because there are 
some principals who know much more than other principals. (CO10) 

It also appeared that the district was not taking full advantage of some of some in-house resources for 
communicating with parents and the community. An online portal made student information easily 
accessible to parents, but utilization rates varied considerably by school, and across the district it 
averaged only 25 percent. Other district communication resources, such as its television station, 
remained underutilized as well. 

In addition to creating the finance, academics, and advisory committees, the district participated in a 
number of partnerships with local businesses and cultural institutions. Through the Adopt-a-School 
program, local businesses partner with individual schools, donating volunteer time for building 
maintenance or tutoring. Both district staff and principals acknowledged that increased support from 
the business community in particular was critical going forward. From a financial standpoint, 
sponsorship of supplemental activities such as athletics or extracurricular activities was a growing need. 
"Fifty six percent of our city is tax-free," one school board member noted. "Let's go to all these people 
that are tax-free and see if we can. ..get them to sponsor our team." (SB02) A district partner also noted 
the importance of ensuring that EPS high school graduates are prepared for work and that local 
companies are willing to hire them— something that required close ties between the district and local 
business. "Tell us what you want. ..what you want us to produce, and we will produce it. But then, by 
God, hire them and give them the carrot, and we don't do that now." (XTP01) 
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Beyond the funding provided through Developing Futures™, GE corporate partners have played a vital 
role in district capacity building efforts. Early in the grant, IT staff from GE Transportation led an 
intensive, collaborative effort to develop and install a new IT platform for the district. "They lived here 
for a year and a half," one central office staff member remarked. (CO09) More recently, process 
engineers from GE helped the central office establish a new science materials center charged with 
sourcing, tracking and maintaining science kits for all schools. GE corporate staff also assisted the 
district's human resources department in revamping its procedures for interviewing and hiring new 
staff, and some GE volunteers worked directly with students through tutoring or mentoring. 

As in many districts, the teachers' union in Erie played a central role in the implementation of reforms. 
Principals, central office staff, and school board members painted a decidedly mixed picture of the 
district's relationship with the Erie Education Association. School board members tended to characterize 
the relationship as generally positive, noting that the district and union had been relatively successful in 
addressing wage and benefit issues over the past decade. Central office staff and principals were more 
measured in their assessments. The union was consulted on key instructional issues such as roll-out of 
the CCSS, and was generally viewed as supportive. Yet this support wavered when reforms turned to 
individual teacher practice or evaluation. When instructional coaches were first introduced, the union 
insisted that coaches only go into classrooms at teachers' request, and that administrators were not 
given access to information about who was working with a coach. One principal suggested that the 
union has since adopted a more collaborative stance regarding coaches. "It was just very secretive. 
Through a process of... meeting and discussing and coming to some agreements that coaching is a good 
support for teachers, we've come way beyond that." (P04) Yet even this year, the union blocked a 
district initiative to recognize a "teacher of the month," objecting to singling out individual teachers 
even for positive reasons. As instructional reforms push further toward both supports and accountability 
measures focused on individual teacher quality, this recalcitrance is a potential concern. 

Indicator 3: Curriculum and Instruction 

Overall, EPS has made great progress towards implementing a cohesive science and mathematics 
curriculum across the district. Of the principals surveyed, approximately 91 percent felt that there was a 
multi-year district plan in place for mathematics. Various interviewees spoke about the shift from 
previous years when "we were a hodgepodge of programs" (POl) or when a person "could go into the 
same school and go from one classroom to another and you would find whatever book happened to be 
what people had on their shelves"(C09) and that was the curriculum the teachers were using. Moving 
away from disparate curricula, staff and principals spoke a common language, with the mathematics and 
science curricula being "inquiry-based", "hands-on", "student-centered" and "collaborative." Some of 
the challenges focused on building or classroom consistency, and science being less of a priority than 
mathematics. Though there were challenges to implementation and assessment, many respondents 
were positive towards the changes. 


Table 3. Curriculum and Instruction 

Curricula are standardized across schools in mathematics. 


3 
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Curricula are standardized across schools in science. 

* 

There is a common approach to mathematics instruction. 

2 

There is common approach to science instruction. 

2 

Teachers have instructional materials (books, kits, lab space) they need to 
carry out instruction. 

3 

Summative assessments are aligned with curriculum and standards. 

1 

Formative assessments guide instruction. 

* 


Note. * indicates insufficient data to make a determination about the prevalence of the given characteristic 


Mathematics 

Most members of central office felt that the mathematics curricula were consistent in elementary and 
middle schools. This coincides with principals perceptions, where 31.8 percent of principals agreed and 
54.5 percent of principals strongly agreed that the district's curriculum frameworks for mathematics are 
specific and clear. Throughout the district, elementary schools (K-5) adopted Everyday Math and middle 
grades (6-8) adopted Connected Mathematics 2 (referred to as CMP2). English as a Second Language 
and Special Education also adopted the same curriculum. The district chose each curriculum and 
developed the instructional frameworks collaboratively and systematically. 

The district formed a mathematics curriculum committee that spent a year researching student 
engagement and learning as specifically in mathematics. The committee then "looked at the programs 
that were out there that met the criteria that we had developed from the research. And then we went 
through the selection process through vendor reviews, development of rubrics, refining rubrics. It was 
quite extensive." (C07) The committee used an innovative approach, the Analysis for Competing 
Hypotheses, to aid in the selection process. A similar process occurred at the high school level, but with 
different results. The committee chose the CME Project and carried over CMP2 as part of their high 
school mathematics curriculum. However, part of the high school mathematics curriculum was to be 
developed in-house and the district had not standardized the curriculum for upper levels of 
mathematics (e.g., trigonometry and calculus). 

While one central office staff member acknowledged that "there are some pockets where people are 
still pulling out old mathematics books," (C09) most respondents mentioned that overall, curriculum use 
was fairly consistent across the system. Less clear was the consistency of instructional practice 
throughout the district. On the one hand, staff noted that teachers had a clear sense of what good 
mathematics instruction should look like. "There is no doubt about, that that has been a priority, that 
teachers have the understanding of what good mathematics instruction looks like, it’s been modeled in 
all the trainings," said one respondent. (C06) On the other hand, not all the schools were implementing 
the new instructional practices. Principal buy-in and ownership of mathematics reform varied from 
building to building. As one principal noted, "I see inconsistencies among the schools, frankly. So, in 
some places there's a strong push and in others there aren't." (P2) 
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Two types of schools appeared more reticent than other schools in implementing new instructional 
practices and inquiry-based curriculum: high-performing schools and high schools. Schools that were 
looking for a new way of instructing students seemed more eager than those schools that were already 
performing well. "Where people's kids struggled and needed different instructional practices, those 
were the sort of ones that got the buy-in much sooner versus the learning academies, so, and those 
[learning academies] were more traditional, because that's what they were used to." (CO10) Since 
students already were doing well in high-achieving schools, teachers did not want to change what 
appeared to be already working. 

In high schools, teachers were concerned about how inquiry-based learning might conflict with their 
classroom management (e.g., having students working on cooperative projects and discussions). There 
were some indications, however, that buy-in was increasing due to high schools working with students 
who had experienced inquiry-based mathematics in middle school. 

While there was ample evidence of district progress in moving toward a coherent instructional system in 
mathematics, EPS faced several challenges in establishing widespread buy-in. One stemmed from the 
lack of clear communication of priorities. Instructional messages that did not come directly from the 
superintendent could sometimes "trickle down" from the central office, and as a result "the teachers 
don't hear it." (C07) In response, the superintendent appeared to be assuming a more prominent role in 
communicating directly to schools and teachers. A second challenge was the tension between 
maintaining or improving scores for the state assessment or fully implementing the new mathematics 
curriculum. Teachers and principals were "getting a mixed message," remarked one central office 
staffer. "'You do what you need to do to make sure they're prepared,' versus, 'I don't care. You do not 
veer away from this curriculum. You will continue to teach it.'" (CO10) At the time of data collection, the 
state was transitioning to Common Core-aligned standards and assessments. EPS struggled to find a 
balance between what was already place and what would be required in the future. 

Science 

While EPS made tentative steps toward an inquiry-based science curriculum prior to Developing 
Futures™, efforts to do so system-wide began when GEF funds were used to purchase science kits for all 
schools. Prior to this initiative, many people noted that science instruction was inconsistent, especially 
at the elementary school level— that only those teachers who were interested in science took the time 
to teach it. Consistency improved in the upper grade bands, more so in high school than middle school. 
For those teachers who taught science, the kits were reported to be instrumental in helping teachers 
adapt. As one principal noted: 

We didn't have a really strong science curriculum before, and [now] to get enough kits 
and to have some direction... I think they [teachers] really, really liked it. And once they 
figured out the set-up, and getting through the initial kit, from what I gathered in 
conversation, what I've seen, that was much smoother. (PI) 
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The science kits also removed one of the major barriers that prevented teachers from teaching science: 
lab preparation. After purchasing science kits, EPS established a material center that refurbished and 
replenished them. "A teacher doesn't have to do one bit of work to do the science," one central office 
staff member noted. "Everything is done for them. They have 25 straws in a bag, they have 25 rubber 
bands. Whatever they need is all done for them, so they've taken that barrier away from 'Oh, I can't do 
this, it's just too much work.'" (C09) 

One way the district tried to improve science instruction was by requiring teachers who wished to use 
the kits to attend professional development on their use. During these sessions, teachers actually 
completed the tasks required by the kits: 

They go and they get these kits, and they go through the standards, and they break 
down the materials themselves and do it themselves. They have discussion collegially 
what are the best ways to do this. And through that discussion, they come out with 
some best practices to take out and then deliver back to the students. (C04) 

Despite the kits and professional development, some respondents noted that not everyone has bought 
in to teaching inquiry-based science, nor have made it a priority at their school. One central office 
member explained that it "depends on the building principal to a large degree and what they focus on, 
whether they buy in to science." (C07) 

Several factors may have hindered the district's progress in establishing a system-wide approach to 
science instruction. First, science was not subject to the same accountability pressure as mathematics 
and ELA. "We still do not have a defined cut score of proficiency," explained one central office staff 
member. "And because we don’t have a cut score of proficiency in science, that gets pushed off to the 
side...AYP is not dependent on science, so we will take the science time and add additional reading and 
math to it." (CO10) 

Second, while access to kits was widespread, that did not necessarily translate to a common vision for 
science instruction. Survey data suggest that principal perceptions of the science curriculum were split, 
with roughly 41 percent of respondents disagreeing that the district's science curriculum frameworks 
are clear and specific. Especially in high schools, science instruction traditionally has been dependent 
upon the initiative of the teacher and any specific grants that the school happened to obtain for science. 
"The reform effort has to be strong and sustained to really reach folks who are isolated," remarked one 
central office staffer. "Even within a science department you're going to have I'm the chemistry teacher, 
I'm the biology teacher, I'm the physics teacher, I'm doing the earth sciences." (C012) 

Assessment 

As noted earlier, among EPS central office staff and principals, one of the biggest areas of concern was 
the lack of alignment between new curricula and state standards, and summative assessments that had 
yet to be updated to reflect those standards. Approximately 41 percent of principals surveyed did not 
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feel that that curriculum, instruction and assessment were well coordinated throughout the district. As 
one principal explained, "there are such specific assessments that we're being held accountable for in 
math that we really can't pull anything out yet, even though we want to, because the kids have to take 
the assessment." (P02) In other words, staying aligned with current state tests actually hindered the 
district's preparation for future ones. 

There were mixed impressions of district formative assessment capacity in mathematics. Some 
participants noted that there were currently no appropriate assessments available. For example 
interviewees said that EPS "outgrew the assessment system" in math (C08), were unable to find a 
district assessment system to meet their needs, and did not have a science assessment that adequately 
gauges growth year to year. Meanwhile, other participants stated that the district either had 
appropriate assessments already, "with the technology that we have, we can actually pinpoint what 
teacher is not following the Everyday Math Curriculum" (SB3), or there were reports that the district 
was exploring some options that could help guide instruction with almost immediate feedback. The 
science kits provided materials which included formative assessments. These kits contained pre- and 
post-tests, created in-house by the curriculum committee, coaches, and science committee to evaluate 
how well students are learning. These tests were provided to the teachers during their professional 
development to use in their classrooms. 

Though not enough data was provided to give a sub-indicator score, it is important to note that there 
appears to be a culture shift towards valuing formative assessments to guide instruction. Many people 
spoke about the desire to have a good enough assessment system that could provide accurate 
information with a quick turnaround. Even in a few buildings there are professional learning 
committees (PLCs) looking at student data and work and making instructional changes. One person in 
the district said, "there are pockets where I feel that's kind of happening, but I still feel like we're not 
there yet. We're moving toward it." (C04) 

Indicator 4. Professional Development for Instruction 

Professional development in EPS has been designed to support the district's inquiry-based and student- 
centered vision for instruction. Initial efforts focused on establishing common approaches to 
mathematics and science instruction, with more recent pushes to support instructional and curricular 
shifts related to the CCSS in mathematics and literacy. Coaching is available, but all coaches are not 
embedded within schools. Areas of concern with regard to professional development included defining 
the roles for instructional support staff such as coaches, as well as the sustainability of efforts funded 
entirely through external resources, including the GEF grant. 


Table 4. Professional Development for Instruction 

PD is aligned with district instructional priorities (content, pedagogical, data). 

3 

There are sufficient resources available to provide the needed PD. 

2 
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School-based PD is available for teachers. 

3 

PD is ongoing. 

3 

PD is data-driven. 

3 

PD is aligned with standards and curricula. 

3 

There is a common understanding of roles played by schools and central office 
with regard to PD. 

2 


Professional development in EPS was a primary strategy for rolling out instructional change initiatives in 
mathematics, science, and literacy and was thus closely aligned with the district's instructional vision. "If 
they wouldn't have had all the PD that they had, there's no way that there would have been successful 
implementation," one principal remarked. "It wouldn't have happened because there was too much of a 
shift." (P 01) In mathematics, professional development efforts were tailored to the specific 
mathematics programs selected for elementary, middle, and high schools, with district staff working 
closely with external partners on designing and delivering the training. In science, professional 
development efforts focused more broadly on inquiry-based instruction, using the newly acquired 
science kits as a vehicle for introducing new instructional practices. 

They get these kits, they go through the standards, and they break down the materials 
themselves and do it themselves. They have a discussion collegially what are the best 
ways to do this. And through that discussion, they come out with some best practices 
to take out and then deliver back to students. (P03) 

A critical distinction between these efforts was that all teachers participated in mathematics 
professional development, while the extent to which teachers participated in science professional 
development depended in large part on how extensively they used the new materials available to them. 
More recently, and with substantial support from GEF, district professional development efforts have 
shifted to preparing for implementation of the CCSS. Central to these efforts was participation by EPS 
staff in the Common Core Institutes hosted in New York City. The institutes catalyzed important 
discussions among the district team about strengths and weaknesses that would influence CCSS 
implementation, and how to build capacity and leverage for implementation. The institute also provided 
a forum for resource sharing among Developing Futures ™ districts with direct impact on 
implementation efforts. A staff member reported, "I went twice last year, learned so much, stole so 
much from [another GEF district], so much of that reflects right in our curriculum maps" (CO 02). 

Local professional development on Common Core focused primarily on literacy. Although the district did 
not charge the literacy department with Common Core professional development explicitly, staff 
suggested that there was no way to implement the Common Core-aligned curriculum without training 
teachers on the standards themselves. 

We couldn't get around the exposure of the Common Core standards, since teachers 
were going to be looking at these maps that had those standards in them. And so, we 
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did some background, but we didn't get into a great deal of depth by virtue of lack of 
time and lack of, really, authority. (CO02) 

All teachers participated in approximately three days of Common Core-focused literacy professional 
development over the course of the year. The sessions were designed and delivered by EPS curriculum 
coordinators and coaches. 

While there was some variation over time and across subjects, in general professional 
development for teachers was designed to model the instructional approaches they were 
expected to employ with students. The sessions worked "almost like a classroom," explained 
one principal. "They open the kit and they go at the project themselves, and what's the best way 
to teach it and make sure you're getting those high-level questions in." (P03) A second feature 
of district-run professional development efforts was that they were sustained over a period of at 
least one year and often longer. "We identified year one, year two, year three. ..and the types of 
training we were going to do, the methodology, the way that. ..teachers and principals and 
district employees were going to be held accountable for this training," one staff member 
explained. (CO07) 

Two primary strategies were used to provide school-level support for district professional development 
efforts: instructional coaching and PLCs. In contrast to "sit and get" professional development events, 
coaches were viewed as effective in the district because of their presence among teachers on a daily 
basis. 

I believe, after having been with this district for so long, the best thing that we've done 
in this district is the K-8 coaching. It has changed practice at the teacher level and at a 
principal level, actually, like nothing I have ever seen. ..They have expertise. ..They' re up 
on research, they know instruction, they deliver professional development, they know 
data. (CO09) 

The coaches who were embedded in schools were more effective in developing trusting relationships 
with teachers, and were able to anticipate the school's needs in a way that external coaches were not. 
Most principals (75 percent) surveyed agreed that mathematics coaches helped improve the quality of 
teaching at their schools, while approximately 60 percent said that science coaches were helpful. 
Coaches played a variety of instructional support roles in their buildings, serving as consultants and 
facilitators for PLCs. In one school, for example, the principal worked with the science and literacy 
coaches to develop a plan for improving writing at his school. This plan involved planning lessons, having 
the principal observe these lessons, and then reviewing student work samples from the lessons. In 
another school, coaches were made available for one-on-one consultations with individual teachers. 

If a teacher feels like they need some extra help, or they want to have someone come 
watch a lesson to just give them constructive feedback, they just request that the coach 
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come into their classroom and give it to them. ..we aren't involved as administration 
...You're not going to get punished from me if you have the coach come in. (P03) 

Coaches who were not embedded in schools faced greater challenges than those who were. One of the 
principals reported that while coaching was an important resource and that the coaches supporting his 
school were highly qualified, talented, and hard working, it would be better if coaches were housed in 
his building. Coaches residing at central had the additional burden of appearing to be arms of 
administration, even though that was not their role. In fact, the lack of clarity about coaches' roles led to 
some problems with teachers' relationships with coaches. Initially, some coaches reported information 
from their classroom visits back to the central office in negative, non-constructive ways. 

Coaching depended on a level of trust between a coach and a teacher.. .The coach 
certainly couldn't be seen as an adjunct of the administration. Whereas when the 
coaching started under the GE grant initially, coaches were identifying teachers that 
were having a hard time, feeding that information right back to the principals, and then 
coaching was being imposed upon teachers as a way to improve their practice. ..there's 
some residual trust issues, there's still a sense that if I have to work with a coach, I must 
be doing something wrong. (CO04) 

The role of PLCs was to give teachers an opportunity to reflect on curriculum, instruction and 
assessment with their peers, in the context of their own practice. Interview data suggest that PLC's 
supported instructional improvement, but were variably implemented across the district. A central 
office staff member articulated a vision of the shift that needs to happen with PLCs for them to more 
strongly support practice, and suggested that most schools had not yet reached that point: 

When we get to the point where I start getting invited to department meetings or PLCs 
where they're huddled around a table looking at student data and making real-time 
instructional changes, when the buildings are not reacting and saying, "We have to do 
some prep for the PSSA exam," when they're actually in September, looking at 
instructional practices. ..and actually making changes well in advance of the threat of the 
state assessment, we'll have changed the culture significantly. (CO04) 

Principals appeared more sanguine about the impact of PLCs. On the survey, most principals (82 
percent) reported feeling prepared to form and utilize teacher teams in their buildings. One described 
evidence of this shift in how teachers learned from one another at their school. "We do videotape 
teachers now and we watch each other teach. And it's okay for the people to sit around the table and go 
'How did that work for you? Did you like that?"' (P03) In another school, teachers worked in their 
inquiry groups with others of the same content area during common planning time to create lessons 
aligned to the CCSS. 


17 


Consortium for Policy Research in Education | cpre.org 



Building District Capacity for System-Wide Instructional Improvement in Erie Public Schools 

While a range of factors dictated the district's professional development focus, there was evidence that 
data from a range of sources was used to plan, modify, or differentiate professional development in the 
central office. For instance, the literacy department saw that writing assessment scores were "abysmal" 
and subsequently focused professional development with teachers on writing pedagogy. Similarly, when 
the mathematics department saw that students were consistently struggling in a particular school, they 
sought out inquiry instruction resources that would help teachers improve. Additionally, feedback forms 
completed as part of the teacher and learning walks provided data for central office staff. 

Survey data suggested that principals were less likely to use data to identify professional development 
needs. Only 18 percent reported that existing data systems were useful for identifying professional 
development needs for their school staff. Whether this reflects limitations in the data available to 
principals, the utility of those data, or lack of principal capacity was not clear from the data. 

While EPS had sufficient resources to support professional development, staff and principals expressed 
concerns about the long-term financial viability of these efforts. Curriculum coordinator and coaching 
positions were funded through supplemental sources (including GEF funds) outside of the district's 
operating budget, raising the question of how these positions would be supported once those funds 
expired. Given the choice between keeping curriculum coordinators and teachers, one central office 
staff member suggested that coordinators should be the priority. "I need the curriculum coordinators' 
positions to stay. Without them a lot of things would just fall apart." (CO09) A principal agreed, 
indicating that the support provided by the central office through the coordinators was valuable and 
highly responsive. 

In addition to funding professional development, making time for the kind of professional development 
needed for the deep change required by the CCSS was a frequently cited concern. For example, the 
literacy department had only one day to train teachers on the CCSS and Understanding by Design before 
rolling them out. Another respondent stated, "There never seems to be enough time to do anything. 
There's no money to pay people to stay after school or anything" (XTP 01). Resources are needed to 
train teachers who have switched grade levels due to severe budget cuts, "maintain the number and 
quality of offerings" for science professional development, and provide time for teachers to collaborate 
and plan together, especially around the CCSS work. As one respondent described, 

...you always get more out of a conversation when you feel like you're more 
comfortable in it. I don't think they feel that when they're at training and in their grade- 
level meetings because it's very structured, there's time constraints. (CO06) 

The district has attempted to maintain the extensive professional development instituted with the GEF 
resources in the face of central office reorganization and a severe budget crisis. School-based coaching, 
while costly, has been a well-received and effective mechanism for pushing curriculum and instruction 
reforms that GEF has funded. 
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lndicator5: Professional Development for Leadership 

Due to a limited number of interviews during which professional development for leadership was 
discussed in detail, there is limited evidence of district capacity in this area. This section describes the 
available qualitative evidence on professional development for leaders along with principal survey data. 
Due to the limitations of the qualitative data, however, we were unable to generate specific ratings for 
this indicator. 

Professional development opportunities are available for school and central office staff to develop 
leadership skills in EPS. Seventy-three percent of principals surveyed felt that the central office takes an 
interest in the professional development of administrators. More than half reported that within the past 
two years they had participated in professional development focused on articulating a mission and 
promoting shared decision-making, and coaching teachers. Roughly three quarters reported 
participating on training focused on what students should know and be able to do in mathematics; 
about half said the same for literacy and science. 

Professional development for school leaders was developed and led by both central office staff and 
external partners, and addressed a wide range of skills, including coaching teachers, recognizing best 
instructional practices, using data, and managing schools. The district provided training in skills that 
support school leadership such as instructional effectiveness, data utilization, and teacher observation 
and evaluation. Through resources provided from GEF, a cohort of 30 school and district staff (building 
administrators, teachers, district staff, and union representatives) participated in training on recognizing 
effective instruction and student engagement in the classroom and how to use data to improve 
instruction. Its purpose was to build a common vision of instruction and rigor among participants. Newer 
building administrators were targeted in particular since they tended to be leading those schools in the 
district with the highest needs. While viewed as effective, there were no explicit plans for sustaining or 
scaling up this training. 

Principals were also trained on recognizing effective content pedagogy in mathematics and science 
through "learning walks" led by district curriculum coordinators. These walks helped principals learn the 
instructional frameworks and identify the types of teacher practices and student behaviors they should 
see. "[Principals] are much more savvy now when they walk through the classroom to say, 'Well, wait a 
minute. You're supposed to be, this is Everyday Math, and why am I seeing those materials? Why am I 
not seeing these materials?'" explained one staff member. (CO09) 

The walks also functioned to build a consistent approach to instruction across classrooms and schools. 
Mathematics learning walks appeared to be well established, with literacy and science developing. One 
principal found learning walks to be a useful source of data. "Walk-throughs, to me, is where it’s 
at. ..That's when you know it's really happening in your building, those two or three minutes you pop in 
here or there." (P03) 
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In addition to the learning walks, principals received three days of training from a local university on 
teacher observations and evaluations. One principal said the training: 

forced me to think about my teachers as possibly unsatisfactory, basic, 
proficient and distinguished, and then ask myself, "Well, how can I move those 
basic teachers into the proficient area? And how can I move those proficient 
teachers into the distinguished area?" And by being forced to develop an action 
plan, which I had never done before, probably never would have even thought 
of... (P04) 

Aside from the instructional leadership cohort mentioned previously, little other leadership professional 
development was reported for central office staff. One staff member reported a lack of such 
opportunities. "There's no mechanism for me, for me personally, to get good feedback, to grow, to 
develop leadership skills. All of that is completely missing. And that I've seen across the board." (CO07) 
To support data-driven decision-making, principals were given training on data utilization and 
customization. The goal was to enable them to utilize data that is most relevant to their building's 
priorities. However, principals' ability to work with this data was limited by a lack of familiarity with 
technology. Thus, some training time had to be spent on teaching the basics of using the computer 
before moving on to instruction on using the data warehouse. 

We have principals that really have not embraced technology, have downloaded this 
responsibility to, say, an assistant principal or somebody else in the school, and they've 
never gotten online. So that was, when we had the training, it was almost like we had 
some beginners in the classroom, not even with the data warehouse, but just with 
signing onto a computer. (CO08) 

There was no clear evidence that a formal system is in place explicitly designed to prepare school or 
central office staff for leadership positions. District and school staff recognized a need to groom future 
leaders, but relatively recent turnover in the central office and severe budget cuts have strained the 
capacity of leaders to respond to the many needs of the district. Many staff members have been given 
responsibilities well outside of the scope of their formal positions. 

You won't find the way we are based on a rational, long, thought-out process. But it 
hasn't escaped us that we have about a year and a half before there will be more 
movement and to start grooming and to start looking at a bench, that there are a lot of 
young administrators that are pretty good... it's just a question of finding the time to do 
it, given all the other things. (CO03) 

To help find a solution, the GEF connected EPS with the District Management Council, a consulting 
organization that helps build management capacity in school districts. They assisted the district with 
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assessing existing and future organizational capacity, defining position roles, and developing a plan to 
meet these needs. At the time of interviews, the central office was in the process of defining roles. 
Despite no formal system for leadership development, some principals reported taking a proactive 
interest in developing leadership among personnel in their buildings. For teachers in the building that 
excel, this principal said, "We need to do more there. I try to find venues to create leadership 
opportunities. That's really important for me." 

Indicator 6: Management Capacity 

Prior to Developing Futures™, EPS was characterized by a low level of management capacity. 

District data and information were not centralized or easily accessible— in some cases they were 
not even electronic. District hiring practices were non-transparent, based primarily on 
relationships and local politics. Seniority drove teacher assignment, and staff evaluations were 
pro-forma at best. In recent years the EPS has made significant upgrades to its data 
infrastructure and has begun the hard work of modernizing its talent management practices, 
though much work remains in these areas. 


Table 5. Management Capacity 

IT infrastructure to collect data is in place. 

2 

IT infrastructure makes data accessible for use. 

2 

There is a systematic or strategic approach to allocating resources. 

2 

HR infrastructure identifies talent effectively. 

1 

Central Office is effective in attracting strong candidates to teaching positions. 

* 

There is a system in place that fills in open positions in a timely manner. 

* 

Teacher evaluations are aligned with instructional expectations. 

2 

Principal evaluations are aligned with instructional expectations. 

* 

Central office evaluations are aligned with instructional expectations. 

* 


Note. * indicates insufficient data to make a determination about the prevalence of the given characteristic 


Since the inception of Developing Futures™ in Erie, the school district has made significant progress in 
making student demographic and academic data accessible to district staff and principals, transitioning 
from a paper-based system in which data were housed in different offices to an integrated, electronic 
system accessible and sortable by a wide range of academic and demographic criteria. At the time of 
data collection, the district's data warehouse contained five years worth of data including student 
grades, attendance, demographics, assessment scores (state tests, etc.) disciplinary information, and 
enrollment. A dashboard function used a color-coded system (red, yellow, green) to easily identify 
students, grades, or schools where a particular indicator raises concerns. For example, if absenteeism 
were high among tenth graders in a particular school, the dashboard would mark that grade in red. The 
district's aim was to use these data to diagnose classroom-level or leadership-level problems at 
struggling schools. According to one central office member, "If a school is not performing well, we'll use 
some of the data from the data warehouse to pick those schools and to call them in and find out what 
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the story is." (CO08) At the time of data collection, the dashboard was only available to district 
administrators and building principals, although there were plans to create access for teachers as well. 

The next phase of the district's plan was to transfer human resource data online in order to connect 
student data with teacher certification and attendance information. Additionally, the tracking of 
professional development opportunities, attendance, and Act 48 credits was not yet part of the data 
warehouse. District staff described this as a point of emphasis in the future, along with developing a tool 
for getting feedback from teachers regarding the usefulness of specific professional development 
sessions. Finally, EPS was in the process of upgrading its infrastructure for payroll, personnel, and 
finance, which will help to streamline administrative functions, including the process for creating 
purchase orders for schools. 

While the infrastructure in place has helped improve access to data, the extent to which principals and 
other district staff efficiently used the data systems depended on their own individual comfort level with 
technology. During initial trainings provided by the district to principals, some principals were 
challenged by the technology. Another potential limitation was principals' facility in making sense of 
data from a range of different sources. Survey data suggest that principals used the data system to 
analyze student learning data on an ongoing basis— 74 percent reported that the data system was 
"useful" or "very useful" for this purpose. On the other hand, only half of all principals reported that the 
system was more than "a little useful" for helping teachers tailor instruction to meet student needs, and 
less than half said so for a variety of other important functions, including guiding curricular adjustments 
(36 percent), evaluating teacher performance (41 percent), informing decisions about resource 
allocation (46 percent), or identifying staff professional development needs (18 percent). Given the 
district's emphasis on making data accessible to leaders, it is perhaps concerning to see indications that 
it may not be fully incorporated into school leaders' planning or decision making. 

The availability of sufficient resources to support both operations and reform efforts is an ever-present 
challenge in Erie. In addition to concerns about the sustainability of professional development efforts 
described earlier, EPS endured successive rounds of painful budget cuts in recent years— a result of 
state budget cuts and a limited local tax base. This combination of pressures led some central office staff 
to conclude that to be effective, the district needed to become more efficient and purposeful with the 
resources it had: 

We cut to the point where our class sizes are really large right now, so in some of our 
schools that serve populations that have more challenges, do we have resources to 
adequately support them? Probably not. But I just don’t see a lot of money coming to us 
on the horizon from the state. Sixty some percent of our budget is state money. So, 
we’ve got to try to find ways to do it within the human and other resources that we 
have. But there’s still, I think, I always feel about our district that we have the potential 
to do a lot within the bounds of our current resources. We just haven't done it. There's 
just, I hate to say low-hanging fruit, but there are just some things that we could do 
differently that I think will yield some real gains. (CO04) 
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To date, most district finance efforts have focused on improving efficiency, with expected savings from 
consolidating IT platforms and bringing network services in-house rather than contracting with an 
external provider. The district was also eyeing savings through facilities consolidation. According to a 
school board member, a study commissioned by the district found that "we have way more space than 
we do have people," (SB02) and that consolidation could save the district up to $50 million. At the time 
of the interviews, the district had not yet decided on a plan for how to accomplish this. 

In addition to seeking greater efficiency, EPS was beginning to be more strategic in its overall approach 
to resource allocation, focusing on how principals could be given greater discretion over school budgets. 
One principal described the genesis of these efforts. 

There was no written policies or procedures [related to budgeting for schools]. People 
didn't know why we had always done it the way we had done it. People would send me 
to another person, who would send me to another person. ..That's when Jay and I 
started having this conversation. And he also had seen that there was a gap there and 
issues there, and that now we're going through this huge budget crisis, and he said that 
he wanted to go towards site-based eventually. And so, I supported that. And so, the 
two of us have been, he gave me some books on principal management of, so I kind of 
piloted some things at my school, again, and I did budget lessons with my faculty. I'm 
giving them much more ownership at department meetings as well, too. And so, I'm 
kind of piloting stuff there as well as training." (P02) 

Overall, it appeared that recent district emphasis on budgeting and resource allocation had been driven 
by the need to cut spending and find efficiencies. Efforts to align resources with strategic focus such as 
site-based budgeting were still in the early stages of development. 

Like IT, talent management and human resources is another area in which EPS had very limited capacity 
prior to the GEF grant, but has made meaningful progress in recent years. One central office staff 
member described a notable shift from a system based on relationships to one based on qualifications. 

We actually give people interviews now. We actually have a process. We actually have, 

I'll even step further back, we actually have everyone going through HR for the first 
time. We had people working in the district that just got hired, just showed up on 
payroll one day, when never having gone through HR, ever. So, everybody goes through 
HR, everybody, down to custodians. That's never happened before, they actually go 
through a process— interviewing process, a whole process. That's all new. (CO09) 

Many respondents spoke of the hiring process for instructional coaches as a turning point. Rather than 
being based on seniority, coaches were hired based on the recommendation of an interview team 
comprised of school, district, and external stakeholders. While coach roles have varied some across 
sites, interview respondents indicated that in general those hired for the positions were well-qualified. 
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In contrast, the hiring of teachers remains primarily seniority-driven. While principals were responsible 
for interviewing and rating prospective teachers, final hiring decisions and school placements were 
made by the central office, with assignments determined first by seniority and then by rating. Principals 
have more control over assignment of teachers within their buildings. "I don't know that good teachers 
are always matched with the kids that they should be matched with," one principal remarked. "I've 
made a couple of teacher switches where I've moved teachers around the building." (P04) This same 
principal raised a concern regarding the lack of minority teaching candidates that the district attracts. He 
partially attributed this to the fact that local colleges did not seem to have a high number of minority 
teaching candidates. 

While EPS was working toward developing HR and talent management systems focused on hiring and 
promoting the best possible candidates for district jobs, some central office staff and partners noted 
that the system has a long history of political patronage and appointments, and that moving to a merit- 
based system represented a radical departure from the status quo. As a result, progress was slow. "It 
used to be— it used to be— that it was not necessarily what you know, but who you know," one district 
partner reflected. (XTP01) "We don't recruit, we politically appoint," a central office staff member 
explained. As a result, the district did not yet have the talent it needed to carry out core functions: 

It's a rarity that you find someone in a position that's truly an expert in that position, 
and you can find a few of those, and when you do they're not heralded as in, "Oh, let's 
see if we can talk to so-and-so because she or he would know, because she or he is an 
expert." That person is shunned and not an expert in their own kingdom, which is really 
sad. (CO07) 

In sum, upgrading talent management systems and practices in EPS appeared to be as much a cultural 
and political challenge as a technical one. The hiring process for coaches represented a small step in the 
right direction, but overall much work remained to move from a system based on connections to one 
based on merit. 

The district is working to refine its teacher evaluation system, bringing it more closely in line with 
instructional expectations. It appeared that progress had been made on developing a framework for 
such evaluations, while finding ways to incorporate them into formal accountability systems remained a 
work in progress. Additionally, the consistency and fidelity with which formal evaluation systems were 
implemented was a concern for some principals and central office staff. Still, survey data revealed that 
82 percent of principals agreed or strongly agreed that the way in which teachers are evaluated is fair. 

EPS's current formal evaluation system is the Peers model used by Charlotte Danielson. Principals were 
trained on this model as part of the district's push to have principals become instructional leaders. The 
observation cycle for teachers includes two formal evaluations per year, plus informal principal walk- 
throughs. Alternatively, teachers with at least three years in their current position or three years in their 
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tenure have the option to do a project. As one principal described, teacher evaluations rely on evidence. 
"It's evidence-based. You do, depending on where they're at in their career, it depends on how many 
formal observations you have to do, and all your evidence and data determines that's satisfactory or 
unsatisfactory that you do." (P03) 

As previously mentioned, EPS also implemented principal walk-throughs or learning walks, partially to 
help principals become more knowledgeable about what teachers were actually doing inside of their 
classrooms. One central office staff member expressed this desire to know "what are the adults doing in 
that school and in the district," (CO09) in order to better understand what teachers are doing to help 
prepare students for assessments. The walkthroughs are one mechanism designed to do this, but, 
according to this same respondent, walkthroughs are not being implemented "100 percent." Similarly, 
principals varied in the extent to which they appeared to be directly engaged with instructional 
improvement within their buildings. Some relied much more on the coaches to provide teachers with 
feedback and support, while others took on a more active role. "We're all over the place. It depends on 
the individual building principals, where they are, what their initiatives in their buildings are, how 
supportive they are of the current work." (CO10) 

Principals suggested that the expectations regarding evaluations were clear to teachers— including 
what they were looking for during walk-throughs. One principal explained the approach for working with 
teachers that are not meeting expectations. The process began with a summation of critical feedback 
from the principal, which was followed by a walkthrough to determine if changes were being made. If no 
improvement was apparent, an Awareness Plan through the Peers system was created; this was 
described as a supportive rather than punitive measure. If the subsequent observations failed to show 
improvement, only then would the principal move toward establishing an unsatisfactory rating. The 
process was described as slow, but fair. (P03) One school board member suggested that there was a 
growing recognition that the system needed to get better at counseling out ineffective staff. 

I think we're getting better at recognizing that we have some people in the district that 
are simply not good educators, and they're engaging the coordinators of math, science 
and literacy to work with the principal, to help them get a teacher back on the right 
path, or making a decision that, "Maybe this isn't the career for me." We're getting 
better at it. (SB03) 

On the other end of the spectrum, the district did not appear to be particularly effective at recognizing 
or rewarding high performers. On the survey, 91 percent of principals strongly disagreed or disagreed 
with the statement that there were clear rewards for high-performing teachers in the district, and 95 
percent said the same for high-performing principals. Ironically, one barrier to recognizing high 
performance appeared to be the teachers' union itself. As noted earlier in this report (Indicator 2), when 
the superintendent tried to launch a "teacher of the month" program to recognize high performers, the 
union objected to singling out any of its members, even for positive reasons. 
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Indicator 7: Evaluation 

Given its historically low capacity for accessing and analyzing data, it is perhaps not surprising 
that evaluation capacity at EPS was a work in progress. As discussed earlier, in recent years the 
central office has more closely tracked student performance, particularly on year-end 
assessments. Yet there was far less evidence that specific metrics or indicators were identified 
and matched with particular reform initiatives, which in turn limited the ability of evaluation 
efforts to inform decision making. 


Table 6. Evaluation 

Specific metrics or indicators are identified for major district initiatives. 

2 

Progress on initiatives is regularly monitored through these indicators (even if 
data is not produced). 

2 

District decisions about stopping, continuing, or expanding initiatives are 
based on evaluation. 

2 


Interview respondents spoke about a number of different ways in which the district was tracking 
progress on different initiatives. Central office leaders conducted a retrospective review of district 
reform efforts to identify lessons that could be incorporated into future work. The group began 
discussing how implementation of past reforms had historically worked in the district, and what made 
some reforms more successful than others. Data was also collected via a survey to district staff 
regarding their knowledge and awareness of the district's initiatives. 

The central office appeared to be adept in collecting data for the purposes of needs assessment and 
program design. Central office members often designed ways to collect data from schools to inform 
their work. One central office staff member described a survey sent to teachers about their progress and 
needs in implementing the literacy block. Another central office member related a story about visiting a 
classroom to get feedback from students on the new science curriculum. 

It was less clear that the central office had in place any mechanism for making decisions about 
expanding or discontinuing programs based on performance. The extent to which evaluation was 
designed into reform initiatives varied considerably. Some reforms, such as the newly adopted 
mathematics programs, were monitored closely using both interim and year-end assessment data. 
Others, such as the coaching initiative, received considerable attention from the central office but did 
not appear to have a formal evaluation attached to them. For example, there was no system in place to 
evaluate the impact of coaches, or to relate the amount of time they spent working with teachers or the 
types of work they did with changes in teacher practice or performance. Given the significant 
investment required to maintain coaches, the lack of a clear mechanism for measuring impact is 
problematic. 
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Conclusion 

In evaluating student performance, researchers and reformers alike are often torn between 
assessing progress relative to absolute standards (proficiency thresholds) and growth over time. 
A similar challenge applies to studying schools and districts. Using best practice as a benchmark, 
it is clear that in several domains— most notably related to management capacity and 
evaluation — EPS has a long way to go before it becomes a high-capacity district. Other areas, 
such as accountability for instructional quality and leadership development, show signs of 
progress but still fall well short of meeting the system's needs. 

When we consider how the system operated prior to Developing Futures ™ —that is, when we 
focus on its growth and development rather than its performance relative to an absolute 
standard— a more positive picture emerges. The instructional system, once fragmented, non- 
transparent and unaccountable, has become much more clearly articulated. Curriculum 
implementation in mathematics and science is more even and consistent than it ever has been. 
A central office characterized by fragmentation and silos has become more collaborative, 
cohesive, and instructional^ focused. Once scattershot, professional development efforts for 
teachers are now closely aligned with instructional initiatives, require active participation, and 
are supported (albeit somewhat inconsistently) by school-level resources such as coaches and 
PLCs. The transition to common curricula, instructional frameworks, and professional 
development supports represents a sea change for EPS. This added capacity will be especially 
important going forward, as the system moves toward implementation of the CCSS. 
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